fix(routing): tolerate transient nonzero htar/htar_large rc in ship_object #28
+201
−8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…bject HPSS can return a nonzero create exit code even though the tar landed fine (observed on Negishi job 40527505: htar rc=70 under 14 concurrent archive jobs, stdout ended "HTAR SUCCESSFUL"; also reported for htar's own /tmp/HTAR_CF_CHK checksum artifact getting swept into a shard's tar). ship_object no longer treats rc != 0 as automatically fatal: it now checks via `hsi ls` whether the tar (+ .idx) actually landed, and if so falls through to the existing round-trip retrieve + per-file MD5 verify before trusting it -- only raising if that verification also fails. Genuinely missing tars still raise exactly as before. Full (untruncated) create stdout is now written to a per-object debug log instead of only a 4000-char tail in the exception, so a real future failure is diagnosable without reproducing it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 12, 2026
…t size Indexed htar -L silently drops members whose relpath is >= 250 chars (rc=70, "name too long for tar archive- file omitted") - reproduced live on Negishi and the root cause of X0A_Task4_Pendleton_Transcriptomics repeatedly failing on the same overlong Novogene GSEA/KEGG path. htar_large has no such ceiling (confirmed live with a 431-char relpath), so choose_transport now also routes on arcname length (HTAR_ARCNAME_LEN_LIMIT = 240), not just size. As defense in depth, ship_object scans create stdout for htar's/tar's own unambiguous member-omission messages and fails fast with every dropped file named, instead of falling through to the rc=70-tolerant path (PR #28) or waiting on the round-trip verify to surface a bare FileNotFoundError. Generalized beyond the name-too-long case after live-testing showed two more silent-drop modes: htar's "Not readable" skip, and - the one case rc gives zero signal for, confirmed rc=0 on Negishi - tar's own "Cannot open" error under htar_large, whose exit code reflects hsi put, not the tar process that actually failed. Checked for a native htar "abort on skip" setting (no man page; pulled the full -H option list from the binary) and a structural exceptions-list option (-H exfile=) - neither exists/works, confirmed by live testing rather than assumed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
3 tasks
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
ship_object()'s htar/htar_large create step no longer treats a nonzero exit code as automatically fatal. Onrc != 0it checks viahsi lswhether the tar (+.idx) actually landed on Fortress, and if so falls through to the existing round-trip retrieve + per-file MD5 verify before trusting it — only raising if that verification also fails. A genuinely missing tar still raises exactly as before.{stem}_{ts}.htar_create.log) instead of only a 4000-char tail in the exception, so a real future failure is diagnosable without reproducing it.Test plan
tests/test_ship_object_htar_rc.py: rc != 0 + tar confirmed present recovers (round-trip actually runs against real copied bytes); rc != 0 + tar genuinely missing still raises; rc = 0 baseline unaffected.python3 -m pytest tests/— 292/292 passed.Generated with Claude Code